Distributed online adaptive subgradient optimization with dynamic bound of learning rate over time‐varying networks

Adaptive online optimization algorithms, such as Adam, RMSprop, and AdaBound, have recently been tremendously popular they widely applied to address the issues in field of deep learning. Despite their prevalence prosperity, however, it is rare investigate distributed versions these adaptive algorithms. To fill gap, a subgradient learning algorithm over time-varying networks, called DAdaxBound, which exponentially accumulates long-term past gradient information possesses dynamic bounds rates under rate clipping developed. Then, regret bound DAdaxBound on convex potentially nonsmooth objective functions theoretically analysed. Finally, numerical experiments are carried out assess effectiveness different datasets. The experimental results demonstrate that compares favourably other competing